Search for: All records

Many recent state-of-the-art results in language tasks were achieved using compound systems that perform multiple Language Model (LM) calls and aggregate their responses. However, there is little understanding of how the number of LM calls -- e.g., when asking the LM to answer each question multiple times and taking a majority vote -- affects such a compound system's performance. In this paper, we initiate the study of scaling properties of compound inference systems. We analyze, theoretically and empirically, how the number of LM calls affects the performance of Vote and Filter-Vote, two of the simplest compound system designs, which aggregate LM responses via majority voting, optionally applying LM filters. We find, surprisingly, that across multiple language tasks, the performance of both Vote and Filter-Vote can first increase but then decrease as a function of the number of LM calls. Our theoretical results suggest that this non-monotonicity is due to the diversity of query difficulties within a task: more LM calls lead to higher performance on "easy" queries, but lower performance on "hard" queries, and non-monotone behavior can emerge when a task contains both types of queries. This insight then allows us to compute, from a small number of samples, the number of LM calls that maximizes system performance, and define an analytical scaling model for both systems. Experiments show that our scaling model can accurately predict the performance of Vote and Filter-Vote systems and thus find the optimal number of LM calls to make. 

Purpose In spite of ongoing and recent initiatives aimed at broadening participation in engineering, the representation of diverse groups of learners in engineering graduate programs in the USA remains a challenge. Foregrounding the voices of 26 Black male engineering faculty, this study aims to investigate how institutions might recruit and retain more Black men in engineering graduate programs. Design/methodology/approach For this study, inductive thematic analysis was used. Findings The authors show that three themes, namely, representation as an asset, invested mentors and faculty, and supportive peer networks described as the “Vibe” manifest as crucial elements for successful recruitment and retention of Black men in engineering graduate programs. Originality/value These findings are meant to augment the conversation around diversity, equity and inclusion in engineering graduate programs and to address a dearth of published research on the Black male engineering population. This work is also meant to help institutions conceptualize ways to create a “Vibe” that might be transferable to their institution’s sociocultural context. 

Abstract BackgroundThough minoritized undergraduate engineering students earn less than 25% of engineering bachelor's degrees, minority‐serving institutions (MSIs) are leading the way in producing a large percentage of those underrepresented engineering bachelor's degree holders. However, much of the published research about the experiences of underrepresented engineering students occurs within the context of predominantly White institutions. Upon deeper inspection into the apparent success of some MSIs, graduation rates of specific minoritized populations (e.g., Black students) remain critically low. This suggests that there is more to be learned about how to better support Black engineering students' success. PurposeWe explored the experiences of Black undergraduate engineering students at a large public doctoral university with very high research activity. Design/MethodWe used interpretative phenomenological analysis to understand the experiences of eight participants. FindingsWe inductively developed two themes to describe how Black engineering students experience success at a Hispanic‐serving institution, which include building success networks and implementing rules of engagement. ConclusionParticipants enacted their cultural capital to construct their circles of success through the intentional engagement of others, resources, and themselves to realize success. This work sheds light on how Black students describe what it means to be successful in their engineering environment.